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Abstract 

Logic  is  a  powerful  tool  for  analyzing  and  verifying  systems,  including  programs,  discrete  sys¬ 
tems,  real-time  systems,  hybrid  systems,  and  distributed  systems.  Some  applications  also  have  a 
stochastic  behavior,  however,  either  because  of  fundamental  properties  of  nature,  uncertain  envi¬ 
ronments,  or  simplifications  to  overcome  complexity.  Discrete  probabilistic  systems  have  been 
studied  using  logic.  But  logic  has  been  chronically  underdeveloped  in  the  context  of  stochastic  hy¬ 
brid  systems,  i.e.,  systems  with  interacting  discrete,  continuous,  and  stochastic  dynamics.  We  aim 
at  overcoming  this  deficiency  and  introduce  a  dynamic  logic  for  stochastic  hybrid  systems.  Our 
results  indicate  that  logic  is  a  promising  tool  for  understanding  stochastic  hybrid  systems  and  can 
help  taming  some  of  their  complexity.  We  introduce  a  compositional  model  for  stochastic  hybrid 
systems.  We  prove  adaptivity,  cadlag,  and  Markov  time  properties,  and  prove  that  the  semantics 
of  our  logic  is  measurable.  We  present  compositional  proof  rules,  including  rules  for  stochastic 
differential  equations,  and  prove  soundness. 


1  Introduction 


Logic  has  been  used  very  successfully  for  verifying  several  classes  of  system  models,  includ¬ 
ing  programs  [Pra76],  discrete  systems,  real-time  systems  [Dut95],  hybrid  systems  [PlalOa],  dis¬ 
tributed  systems,  and  distributed  hybrid  systems  [PlalOb].  This  gives  us  confidence  in  the  power 
of  logic.  Not  all  aspects  of  real  systems  can  be  represented  faithfully  by  these  models,  however. 
Some  systems  are  inherently  uncertain,  either  because  of  fundamental  properties  of  nature,  be¬ 
cause  they  operate  in  an  uncertain  environment,  or  because  deterministic  models  are  simply  too 
complex.  Such  systems  have  a  stochastic  dynamics.  Nonde  termini  Stic  overapproximations  may  be 
too  inaccurate  for  a  meaningful  analysis,  e.g.,  because  a  worst-case  analysis  would  let  bad  envi¬ 
ronment  actions  happen  always,  which  is  very  unlikely.  Discrete  probabilistic  systems  have  been 
studied  using  logic.  Yet,  complex  systems  are  driven  by  joint  discrete,  continuous,  and  stochastic 
dynamics.  Logic  has  been  chronically  underdeveloped  in  the  context  of  these  stochastic  hybrid 
systems. 

Classical  logic  is  about  boolean  truth  and  yes/no  answers.  That  is  why  it  is  tricky  to  use 
logic  for  systems  with  stochastic  effects.  Logic  has  reached  out  into  probabilistic  extensions  at 
least  for  discrete  programs  [Koz81,  Koz85,  FH84]  and  for  first-order  logic  over  a  finite  domain 
[RD06].  Logic  has  been  used  for  the  purpose  of  specifying  system  properties  in  model  checking 
finite  Markov  chains  [YKNP06]  and  probabilistic  timed  automata  [KNSW07].  Stochastic  hybrid 
systems,  instead,  are  a  domain  where  logic  and  especially  proof  calculi  have  so  far  been  more 
conspicuous  by  their  absence.  Given  how  successful  logic  has  been  elsewhere,  we  want  to  change 
that. 

Stochastic  hybrid  systems  [BL06,  CL06,  HLSOO]  are  systems  with  interacting  discrete,  con¬ 
tinuous,  and  stochastic  dynamics.  There  is  not  just  one  canonical  way  to  add  stochastic  behavior 
to  a  system  model.  Stochasticity  might  be  restricted  to  the  discrete  dynamics,  as  in  piecewise  de¬ 
terministic  Markov  decision  processes,  restricted  to  the  continuous  and  switching  behavior  as  in 
switching  diffusion  processes  [GAM97],  or  allowed  in  many  parts  as  in  so-called  General  Stochas¬ 
tic  Hybrid  Systems;  see  [BL06,  CL06]  for  an  overview.  Several  different  forms  of  combinations 
of  probabilities  with  hybrid  systems  and  continuous  systems  have  been  considered,  both  for  model 
checking  [FTE10,  KR08,  CL06]  and  for  simulation-based  validation  [MS06,  ZPC10]. 

We  develop  a  very  different  approach.  We  consider  logic  and  theorem  proving  for  stochastic 
hybrid  systems1  to  transfer  the  success  that  logic  has  had  in  other  domains.  Our  approach  is 
partially  inspired  by  probabilistic  PDL  [Koz85]  and  by  barrier  certificates  for  continuous  dynamics 
[PJP07].  We  follow  the  arithmetical  view  that  Kozen  identified  as  suitable  for  probabilistic  logic 
[Koz85], 

Classical  analysis  is  provably  inadequate  [KP10]  for  analyzing  even  simple  continuous  stochas¬ 
tic  processes.  We  heavily  draw  on  both  stochastic  calculus  and  logic.  It  is  not  possible  to  present 
all  mathematical  background  exhaustively  here.  But  we  provide  basic  definitions  and  intuition  and 
refer  to  the  literature  for  details  and  proofs  of  the  main  results  of  stochastic  calculus  [KS91,  0ksO7, 
KP10], 

'Note  that  there  is  a  model  called  Stochastic  Hybrid  Systems  [HLSOO].  We  do  not  mean  this  specific  model  in  the 
narrow  sense  but  refer  to  stochastic  hybrid  systems  as  the  broader  class  of  systems  that  share  discrete,  continuous,  and 
stochastic  dynamics. 
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Our  most  interesting  contributions  are: 


1.  We  present  the  new  model  of  stochastic  hybrid  programs  (SHPs)  and  define  a  compositional 
semantics  of  SHP  executions  in  terms  of  stochastic  processes. 

2.  We  prove  that  the  semantic  processes  are  adapted,  have  almost  surely  cadlag  paths,  and  that 
their  natural  stopping  times  are  Markov  times. 

3.  We  introduce  a  new  logic  called  stochastic  differential  dynamic  logic  (ScLC)  for  specifying 
and  verifying  properties  of  SHPs. 

4.  We  define  a  semantics  and  prove  that  it  is  measurable  such  that  probabilities  are  well-defined 
and  probabilistic  questions  become  meaningful. 

5.  We  present  proof  rules  for  Sd£  and  prove  their  soundness. 

6.  We  identify  the  requirements  for  using  Dynkin’s  formula  for  proving  properties  using  the 
infinitesimal  generator  of  stochastic  differential  equations. 

ScLC  makes  the  rich  semantical  complexity  and  deep  theory  of  stochastic  hybrid  systems  accessible 
in  a  simple  syntactic  language.  This  makes  the  verification  of  stochastic  hybrid  systems  possible 
with  elementary  syntactic  proof  principles. 


2  Preliminaries:  Stochastic  Processes 

We  fix  a  dimension  del  for  the  Euclidean  state  space  equipped  with  its  Borel  a-algebra  B , 
i.e.,  the  a- algebra  generated  by  all  open  subsets.  A  a-algebra  on  a  set  0  is  a  nonempty  set  T  C  2n 
that  is  closed  under  complement  and  countable  union.  We  axiomatically  fix  a  probability  space 
(fi,  IF,  P )  with  a  a-algebra  T  C  2n  of  events  on  space  fl  and  a  probability  measure  P  on  P  (i.e., 
P  :  T  — y  [0, 1]  is  countable  additive  with  P  >  0,  P(f2)  =  1).  We  assume  the  probability  space  has 
been  completed,  i.e.,  every  subset  of  a  null  set  (i.e.,  P(A)  =  0)  is  measurable.  A  property  holds 
P-almost  surely  ( a.s .)  if  it  holds  with  probability  1.  A  filtration  is  a  family  (Pfityo  of  a-algebras 
that  is  increasing,  i.e.,  Ps  C  Ff  for  all  s  <  t.  Intuitively,  Tt  are  the  events  that  can  be  discriminated 
at  time  t.  We  always  assume  a  filtration  (Pt)t> o  that  has  been  completed  to  include  all  null  sets 
and  that  is  right-continuous,  i.e.,  Tt  =  Dtot  Xu  for  all  t.  We  generally  assume  the  compatibility 
condition  that  T  coincides  with  the  a-algebra  T*  :=  a  ((Jt>0  P) ,  i.e.,  the  a-algebra  generated  by 
all  Tf 

For  a  a-algebra  E  on  a  set  D  and  the  Borel  a-algebra  B  on  Wl,  function  /  :  I)  — >•  RJ  is  measur¬ 
able  iff  /  1(B)  G  E  for  all  B  G  B  (or,  equivalently,  for  all  open  B  C  M,/).  An  Revalued  random 
variable  is  an  P-measurable  function  X  :  O  — >■  Rr/.  All  sets  and  functions  definable  in  first-order 
logic  over  real  arithmetic  are  Borel-measurable.  A  stochastic  process  X  is  a  collection  {Xt}teT 
of  Md- valued  random  variables  Xt  indexed  by  some  set  T  for  time.  That  is,  X  :  T  x  Q  — »  is  a 
function  such  that  for  all  t  G  T,  Xt  —  X (t,  ■)  :  O  — >•  W1  is  a  random  variable.  Process  X  is  adapted 
to  filtration  (Pt)t>o  if  Xt  is  J7, -measurable  for  each  t.  That  is,  the  process  does  not  depend  on  future 
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events.  We  consider  only  adapted  processes  (e.g.,  using  the  completion  of  the  natural  filtration  of 
a  process  or  the  completion  of  the  optional  cr-algebra  for  T  [KS91]).  A  process  X  is  cadlag  iff  its 
paths  t  i-)-  Xt[pj )  (for  each  u  G  0)  are  cadlag  a.s.,  i.e.,  right-continuous  (lim^t  AT^cu)  =  Xt(oj)) 
and  left  limits  (limS/*t  X^(cu))  exist. 

We  further  need  an  e-dimensional  Brownian  motion  W  (i.e.,  W  is  a  stochastic  process  starting 
at  0  that  is  almost  surely  continuous  and  has  independent  increments  that  are  normally  distributed 
with  mean  0  and  variance  equal  to  the  time  difference).  Brownian  motion  is  mathematically  ex¬ 
tremely  complex.  Its  paths  are  almost  surely  continuous  everywhere  but  differentiable  nowhere 
and  of  unbounded  variation.  Intuitively,  W  can  be  understood  as  the  limit  of  a  random  walk.  We 

denote  the  Euclidean  vector  norm  by  |x|  and  use  the  Frobenius  norm  |er|  :=  \jYhij  f°r  matrices 

or  G  Rdxe. 

3  Stochastic  Differential  Equations 

We  consider  stochastic  differential  equations  [0ksO7,  KP10]  to  describe  stochastic  continuous 
system  dynamics.  They  are  like  ordinary  differential  equations  but  have  an  additional  diffu¬ 
sion  term  that  varies  the  state  stochastically.  Stochastic  differential  equations  are  of  the  form 
dXt  =  b(Xt)dt  +  o(Xt)dWt.  We  consider  Ito  stochastic  differential  equations,  whose  solutions 
are  defined  by  the  stochastic  Ito  integral  [0ksO7,  KP10],  which  is  again  a  stochastic  process.  Like 
in  an  ordinary  differential  equation,  the  drift  coefficient  b(Xt)  de¬ 
termines  the  deterministic  part  of  how  Xt  changes  over  time  as  a 
function  of  its  current  value.  As  a  function  of  Xt,  the  diffusion  co¬ 
efficient  a(Xt )  determines  the  stochastic  influence  by  integration 
with  respect  to  the  Brownian  motion  process  Wt.  See  Fig.  1  for 
two  sample  paths.  Ordinary  differential  equations  are  retained  for 
a  =  0.  We  focus  on  the  time-homogenous  case,  where  b  and  cr  are 
time-independent,  because  time  could  be  added  as  an  extra  state 
variable. 

Definition  1  (Stochastic  differential  equation)  A  stochastic  process 
X  :  [0,  oo)  x  Q  — >  Wl  solves  the  (ltd)  stochastic  differential  equation 

dXt  =  b{Xt)dt  +  a(Xt)dWt  (1) 

with  X0  =  Z,  ifXt  —  Z  +  J  b(Xt  )dt  +  f  o(Xt)dWt,  where  f  a(Xt)dWt  is  an  ltd  integral  process 
[ 0ksO7,  KP10]. 

For  simplicity,  we  always  assume  b  :  Rd  — »  Rd  and  a  :  Rd  — »  Mdxe  to  be  measurable  and  locally 
Lipschitz-continuous: 

ViV30Va;,  y  :  |x|,  \y\  <  N  =>-  | b(x)  —  b(y) \  <  C\x  —  y\,  |cr(x)  —  a(y)\  <  C\x  —  y\ 

As  an  integral  of  an  a.s.  continuous  process,  solution  X  has  almost  surely  continuous  paths 
[0ksO7].  A.s.  continuous  solution  X  is  pathwise  unique  [KP10,  Ch  4.5].  Process  X  is  a  strong 
Markov  process  for  each  initial  value  x  [0ksO7,  Theorem  7.2.4]. 


Figure  1:  Sample  paths  with 
b  =  1  (top)  and  b  =  0  (bot¬ 
tom),  cr  =  1 
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4  Stochastic  Hybrid  Programs 

As  a  system  model  for  stochastic  hybrid  system,  we  introduce  stochastic  hybrid  programs  (SHPs). 
SHPs  combine  stochastic  differential  equations  for  describing  the  stochastic  continuous  system 
dynamics  with  program  operations  to  describe  the  discrete  switching,  jumps,  and  discrete  stochas¬ 
tic  choices.  These  primitive  dynamics  can  be  combined  programmatically  in  flexible  ways.  All 
basic  terms  in  stochastic  hybrid  programs  and  stochastic  differential  dynamic  logic  are  polyno¬ 
mial  terms  built  over  real- valued  variables  and  rational  constants.  Our  approach  is  sound  for  more 
general  settings,  but  first-order  real  arithmetic  is  decidable  [Tar51]. 

4.1  Syntax 

Stochastic  hybrid  programs  (SHPs)  are  formed  by  the  following  grammar  (where  xt  is  a  variable, 
x  a  vector  of  variables,  6  a  term,  b  a  vector  of  terms,  a  a  matrix  of  terms,  H  is  a  quantifier-free 
first-order  real  arithmetic  formula,  A,  v  >  0  are  rational  numbers): 

a  ::=  Xi  :=  9  \  Xi  :=  *  |  ? H  j  dx  =  bdt  +  adW  &  H  \  \a  ©  v/3  \  a;  /3  \  a* 

Assignment  Xi'.—  Q  deterministically  assigns  term  6  to  variable  Xi  instantaneously.  Random  as¬ 
signment  Xi  :  =  *  randomly  updates  variable  Xi,  but  unlike  in  classical  dynamic  logic  [Pra76],  we 
assume  a  probability  distribution  for  x.  As  one  example  for  a  probability  distribution,  we  consider 
uniform  distribution  in  the  interval  [0,1],  but  other  distributions  can  be  used  as  long  as  they  are 
computationally  tractable,  e.g.,  definable  in  first-order  real  arithmetic. 

Most  importantly,  dx  =  bdt  +  adW  &  H  represents  a  stochastic  continuous  evolution  along  a 
stochastic  differential  equation,  restricted  to  the  evolution  domain  region  H,  i.e.,  the  stochastic 
process  will  not  continue  when  it  leaves  H.  We  assume  that  dx  =  bdt  +  adW  satisfies  the  as¬ 
sumptions  of  stochastic  differential  equations  according  to  Def.  1 .  In  particular,  the  dimensions  of 
the  vectors  x,  b ,  matrix  a,  and  (vectorial)  Brownian  motion  W  fit  together  and  b,  a  are  globally 
Lipschitz-continuous  (which  is  first-order  definable  for  polynomial  terms  and,  thus,  decidable  by 
quantifier  elimination  [Tar51]). 

Test  ? H  represents  a  stochastic  process  that  fails  (disappears  into  an  absorbing  state)  if  H  is 
not  satisfied  yet  continues  unmodified  otherwise.  Linear  combination  Act  ©  u/3  evolves  like  a  in 
A  percent  of  the  cases  and  like  f3  otherwise.  We  simply  assume  A  +  v  —  1.  Sequenticd  composition 
ct;  (3  and  repetition  a*  work  similarly  to  dynamic  logic  [Pra76],  except  that  they  combine  SHPs. 

4.2  Stochastic  Process  Semantics 

The  semantics  of  a  SHP  is  the  stochastic  process  that  it  generates.  The  semantics  [a]  of  a  SHP 
a  consists  of  a  function  [a]  :  (O  — »  Rd )  — y  ([0,  oo)  xS©  Md)  that  maps  any  Revalued  ran¬ 
dom  variable  Z  describing  the  initial  state  to  a  stochastic  process  [a] z  together  with  a  function 
(|  a  I)  :  (Q  — >■  W1)  — y  (0  — >■  M)  that  maps  any  Rd-  valued  random  variable  Z  describing  the  initial 
state  to  a  stopping  time  (\a\)Z  indicating  when  to  stop  {a\Z .  Often,  an  ^-measurable  random 
variable  Z  or  deterministic  state  is  used  to  describe  the  initial  state.  We  assume  independence  of  Z 
from  subsequent  stochastic  processes  like  Brownian  motions  occurring  in  the  definition  of  [o  ] Z . 
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For  an  Revalued  random  variable  Z ,  we  denote  by  Z  the  stochastic  process 

Z  :  {0}  x  Fl  — »  Rd;  (0,o;)  (->•  Z0(u;)  :=  Z(u) 

that  is  stuck  at  Z.  We  write  x  for  the  random  variable  Z  that  is  a  deterministic  state  Z(u)  :=  x  for 
all  u  G  Fl.  We  write  [a]'1  and  (| or D :r  for  \a\Z  and  (\a\)Z  then. 

In  order  to  simplify  notation,  we  assume  that  all  variables  are  uniquely  identified  by  an  index, 

i.e.,  the  only  occurring  variables  are  xi,x2 , . . .  ,xd.  We  write  Z(oj)  \=  H  if  state  Z(u)  satisfies 
first-order  real  arithmetic  formula  H  and  Z(u)  \f=  H  otherwise.  In  the  semantics  we  will  use 
a  family  of  random  variables  {Ui}i£j  that  are  distributed  uniformly  in  [0, 1]  and  independent  of 
other  Uj  and  all  other  random  variables  and  stochastic  processes  in  the  semantics.  Hence,  U  sat¬ 
isfies  P({co  G  Fl  :  U (ce)  <  s})  =  with  the  usual  extensions  to  other  Borel  subsets.  To 

describe  this  situation,  we  just  say  that  “U  ~  U((),  1)  is  i.i.d.  (independent  and  identically  dis¬ 
tributed)”,  meaning  that  U  is  furthermore  independent  of  all  other  random  variables  and  stochastic 
processes  in  the  semantics.  We  denote  the  characteristic  function  of  a  set  S  by  ls,  which  is  defined 
by  ls(x)  1  if  x  e  S  and  Zs(x)  :=  0  otherwise. 

Definition  2  (Stochastic  hybrid  program  semantics)  The  semantics  ofSHP  a  is  defined  by 

[a]  :  (f2  — >  Md)  — >  ([0,  oo)  x  Q  — >  Md);  Z  i->  [a]Z  =  ([a]f)t>o 

and 

da|)  :  (fi  — >■  Rd)  -G  (fi  -G  M);  Z  ^  (\a\)Z 

These  functions  are  inductively  defined  for  random  variable  Z  by 

1.  [a;*  :=  0]Z  =  Y  where  Y(u)i  =  and  Yj  =  Zjfor  all  j  f  i.  Further,  :=  9\)Z  =  0. 

2.  \xi  :=  *jZ  =  U  where  Uj  =  Zj  for  all  j  f  i  and  Ut  ~  U{ 0, 1)  is  i.i.d.  and  T ^-measurable. 
Further,  (]xj  :=  *\)Z  =  0. 

3.  pH]z  —  Z  on  the  event  {Z  |=  H}  and  (\?H\)Z  =  0  (on  all  events  u  e  Ft).  Note  that 
is  not  defined  on  the  event  {Z  \fi  IF). 

4.  \dx  =  bdt  +  adW  &  H\Z  is  the  process  X  :  [0,  oo)  xOg!'1  that  solves  the  (ltd)  stochas¬ 
tic  differential  equation  dXt  =  |6|| X' dt  +  [a] XtdBt  with  X0  =  Z  on  the  event  {Z  \=  IF), 
where  If  is  afresh  e-dimensional  Brownian  motion  if  a  has  e  columns.  We  assume  that  Z  is 
independent  of  the  o-algebra  generated  by  (Bt)t>0. 

Further,  (|  dx  =  bdt  +  adW  &  II\)Z  =  inf{t  >  0  :  Xt  f  IF).  Note  that  X  is  not  defined  on 
the  event  {Z  \f=  H }. 


[Aa  ©  ufif  =  lu^laf  +lu>xl/3f  = 

dAa  ©  ufi\)z=lu<x(\a\)Z+Iu>MZ 

where  U  ~  U({),  1)  is  i.i.d.  and  Xu-measurable. 


on  the  event  {U  <  A} 
on  the  event  {U  >  A} 
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6. 


z 

t 


u.  /  i  jj  i 


on  {t  <  (\a\)Z} 
on  {t  >  (\a\)Z} 


and 


dof  + 


7.  \a*ft  =  \otn\Z  on  the  event  {(|an|)Z  >  t}  and  da*l)Z 


where  a0  =  ? true,  a 1  =  a,  and  an+1  =  a;  an. 


lim  (\an\)Z 

n— >-oo 


For  Case 7,  note  that  (| an\)Z  is  monotone  in  n,  hence  the  limit  (|a*|)z  exists  and  is  finite  if  the 
sequence  is  bounded.  The  limit  is  oo  otherwise.  Note  that  [a*]f  is  independent  of  the  choice 
of  n  on  the  event  { (] on |) 7  >  t]  (but  not  necessarily  independent  of  n  on  the  event  {(\an\)z  >  t}, 
because  a  might  start  with  a  jump  after  an ).  Observe  that  [a*]f  is  not  defined  on  the  event 
{Vn  (|o'r'  |)  <  t } ,  which  happens,  e.g.,  for  Zeno  executions  violating  divergence  of  time.  It  would 
still  be  possible  to  give  a  semantics  in  this  case,  e.g.,  at  t  =  (|o”l)  ,  but  we  do  not  gain  much  from 
introducing  those  technicalities. 

In  the  semantics  of  \a\z ,  time  is  allowed  to  end.  We  explicitly  consider  [a]f  as  not  defined  for 
a  realization  u  if  a  part  of  this  process  is  not  defined,  because  of  failed  tests  in  a.  The  process  may 
be  explicitly  not  defined  when  t  >  (fD  .  Explicitly  being  not  defined  can  be  viewed  as  being  in  a 
special  absorbing  state  that  can  never  be  left  again,  as  in  killed  processes.  The  stochastic  process 
l a]z  is  only  intended  to  be  used  until  time  (\a\)z .  We  stop  using  [a] ^  after  time  (laj)^. 

A  Markov  time  (a.k.a.  stopping  time)  is  a  non-negative  random  variable  r  such  that  {t  <  t}  G  Tt 
for  all  t.  For  a  Markov  time  r  and  a  stochastic  process  Xt,  the  following  process  is  called  stopped 
process  XT 


x r  :=  xtnr  = 


if  t  <  t 
if  t  >  t 


where  t  fl  r  :=  min{t,  r} 


A  class  C,  of  processes  is  stable  under  stopping  if  X  e  C  implies  XT  e  C  for  every  Markov  time 
r.  Right  continuous  adapted  processes,  and  processes  satisfying  the  strong  Markov  property  are 
stable  under  stopping  [Dyn65,  Theorem  10.2]. 

Most  importantly,  we  show  that  the  semantics  is  well-defined.  We  prove  that  the  natural  stop¬ 
ping  times  (\a\)z  are  actually  Markov  times  so  that  it  is  meaningful  to  stop  process  {a}z  at  (\a\)z 
and  useful  properties  of  \a\z  inherit  to  the  stopped  process  |Q:]/ri:|0:|^.  Furthermore,  we  show  that 

the  process  \a\z  is  adapted  (does  not  look  into  the  future)  and  cadlag,  which  will  be  important  to 
define  a  semantics  for  formulas.  We  give  a  proof  of  the  following  theorem  in  Appendix  A.  1. 


Theorem  1  (Adaptive  cadlag  process  with  Markov  times)  For  each  SHP  a  and  any  -valued 
random  variable  Z,  \a\Z  is  an  a.s.  cadlag  process  and  adapted  (to  the  completed  filtration  (Xt)t>  o 
generated  by  Z  and  the  constituent  Brownian  motion  ( Bs)s<t  and  uniform  U  processes)  and  (| a  |) 7 
is  a  Markov  time  (for  (  7r/)/,>o).  In  particular,  the  end  value  [a:]^z  is  again  (F^z -measurable. 

Note  in  particular,  that  the  event  {(|an|)  >  t}  is  ^-measurable,  thus,  by  [KS91,  Prop  1.2.3],  the 
event  {dan|)Z  >  t}  in  Case 7  of  Def.  2  is  ^-measurable.  As  a  corollary  to  Theorem  1,  [ ajz  is 
progressively  measurable  [KS91,  Prop  1.1.13]. 
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5  Stochastic  Differential  Dynamic  Logic 

For  specifying  and  analyzing  properties  of 
SHPs,  we  introduce  stochastic  differential  dy¬ 
namic  logic  Sd£. 

5.1  Syntax 

Function  terms  of  stochastic  differential  dy¬ 
namic  logic  Sd£  are  formed  by  the  gram¬ 
mar  (F  is  a  primitive  measurable  function  de¬ 
finable  in  first-order  real  arithmetic,  e.g.,  the 
characteristic  function  Xs  of  a  measurable  set 
S  definable  in  first-order  real  arithmetic,  B 
is  a  boolean  combination  of  such  characteris¬ 
tic  functions  using  operators  A,  V,  -i,  — *  from 
Fig.  2,  A,  v  are  rational  numbers): 

f,9  ■■=  F\xf  +  "9  1  Bf  I  («)/ 

These  are  for  linear  (A/  +  vg)  or  boolean  product  ( Bf )  combinations  of  terms.  Term  ( a)f  rep¬ 
resents  the  supremal  value  of  /  along  the  process  belonging  to  a.  The  syntactic  abbreviations  in 
Fig.  2  can  be  useful.  Formulas  of  Sd£  are  simple,  because  Sd£  function  terms  are  powerful.  Sd£ 
formulas  express  equational  and  inequality  relations  between  Sd£  function  terms  /,  g.  They  are  of 
the  form: 

<f>  ::=  /  <  9  |  /  =  9 


0  =  X) 

1  =  FRd 
-/  =  !-/ 

A  A  B  =  AB 
Ay  B  =  A  +  B  -  AB 
A^B=1-A  +  AB 

if(tf)  {«}else{/5}  =  ^  (?#;«)  ©  ^  (?->#;£) 
while  (IT)  {a}  =  (?iF;  a )*;  ?-> H 

[oi\f  = 

Figure  2:  Common  Sd£  and 
SHP  abbreviations 


5.2  Measurable  Semantics 

The  semantics  of  classical  logics  maps  an  interpretation  to  a  truth-value.  This  does  not  work  for 
stochastic  logic,  because  the  state  evolution  of  SHPs  contained  in  Sd£  formulas  is  stochastic,  not 
deterministic.  Instead,  we  define  the  semantics  of  an  Sd£  function  term  as  a  random  variable. 

Definition  3  (Sd£  semantics)  The  semantics  [/]  of  a  function  term  f  is  a  function 

[/]  :  (ft  -A  Rd)  — ►  (ft  -A  R) 

that  maps  any  Wl -valued  random  variable  Z  describing  the  current  state  to  a  random  variable 
[ f]z.  It  is  defined  by 

1.  \F\Z  =  Fe(Z),  i.e.,  |F]z(o;)  =  Fe(Z(u>))  where  function  F  denotes  F£ 

2-  [A/  +  ^]Z  =  \{f\Z  +  v\g\z 

3.  \Bf]Z  =  {B]Z  *  l f\z ,  i.e.,  multiplication  [Bfjz(u)  =  {B]z (u)  *  \f\Z (u) 
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4-  =  sup{[/]Nt  :  0  <t<  (\a\)z} 

When  Z  is  not  defined  (results  from  a  failed  test),  then  [fjz  is  not  defined.  To  avoid  partiality,  we 
assume  the  convention  [ f\z  :=  0  when  Z  is  not  defined. 

If  /  is  a  characteristic  function  of  a  measurable  set,  then  [(a)/]z  corresponds  to  a  random 
variable  that  reflects  the  supremal  /  value  that  a  can  reach  at  least  once  during  its  evolution 
until  stopping  time  (|a|)  when  starting  in  a  state  corresponding  to  random  variable  Z.  Then 
P([(a)/Jz  =  1)  is  the  probability  with  which  a  reaches  /  at  least  once  and  E({(a)f}z)  is  the 
expected  value,  given  Z.  This  includes  the  special  case  where  Z  is  a  deterministic  state  Z(u)  :=  x 
for  all  w  e  0.  But  first,  we  prove  that  these  quantities  are  well-defined  probabilities  at  all.  Note 
that  well-definedness  of  the  definition  in  case  4  uses  Theorem  1. 

Cases  1-3  of  Def.  3  are  as  in  [Koz85]  with  the  notable  exception  of  case  4,  which  we  define 
as  a  supremum,  not  an  integral.  The  reason  is  that  we  are  interested  in  probabilistic  worst-case 
verification,  not  in  average-case  verification.  For  discrete  programs,  it  is  often  sufficient  to  consider 
the  input-output  behavior,  so  that  Kozen  did  not  need  to  consider  the  temporal  evolution  of  the 
program  over  time,  only  its  final  (probabilistic)  outcome  [Koz85].  In  stochastic  hybrid  systems, 
the  temporal  evolution  is  highly  relevant,  in  addition  to  the  stochastic  behavior.  When  averaging 
over  time,  the  system  state  may  be  very  probably  good  (the  integral  of  the  error  is  small).  But, 
still,  it  could  be  very  likely  that  the  system  exhibits  a  bug  at  some  state  during  a  run.  In  this  case, 
we  would  still  want  to  declare  such  a  system  as  broken,  because,  when  using  it,  it  will  very  likely 
get  us  into  trouble.  Stochastic  average-case  analysis  is  interesting  for  performance  analysis.  But 
for  safety  verification,  supremal  stochastic  analysis  is  more  relevant,  because  a  system  that  is  very 
probably  broken  at  some  time,  is  still  too  broken  to  be  used  safely.  We  thus  consider  stochastic 
dynamics  with  worst-case  temporal  behavior,  i.e.,  our  semantics  performs  stochastic  averaging  (in 
the  sense  of  probability)  among  different  behaviors,  but  considers  supremal  worst-case  probability 
over  time.  The  logic  Sd£  is  intended  to  be  used  (among  other  things)  to  prove  bounds  on  the 
probability  that  a  system  fails  at  some  point  at  all. 

A  car  that,  on  average  over  all  times  of  its  use,  has  a  low  crash  rate,  but  still  has  a  high  proba¬ 
bility  of  crashing  at  least  once  during  the  first  ride  would  not  be  safe.  This  is  one  example  where 
stochastic  hybrid  systems  exhibit  new  interesting  characteristics  that  we  do  not  see  in  discrete 
systems. 

We  show  that  the  semantics  is  well-defined.  We  prove  that  [/ ]z  is,  indeed,  a  random  variable, 
i.e.,  measurable.  Without  this,  probabilistic  questions  about  the  value  of  formulas  would  not  be 
well-defined,  because  they  are  not  measurable  with  respect  to  the  probability  space  (0,  T .  P )  and 
the  Borel  cr-algebra  on  M. 

Theorem  2  (Measurability)  For  any  M.d -valued  random  variable  Z,  the  semantics  {f}Z  of  func¬ 
tion  term  f  is  a  random  variable  (i.e.,  T -measurable). 

We  give  a  proof  of  this  theorem  in  Appendix  A. 2. 

Corollary  1  (Pushforward  measure)  For  any  Wl -valued  random  variable  Z  and  function  term 
f,  probability  measure  P  induces  the  pushforward  measure 

s  ^  Pmf)-'(S))  =  P({u  e  a :  Iff  M  e  S})  =  P(i/f  e  S) 
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which  defines  a  probability  measure  on  BL  Hence,  for  each  Borel-measurable  set  S,  the  probability 
P([/]Z  G  S)  is  well-defined. 

We  say  that  f  <  g  is  valid  if  it  holds  for  all  Revalued  random  variables  Z: 

1=  /  <9  iff  for  a11  Z,  Iff  <  Idf ,  i-e.j.  ([/]Z)M  <  ([s]Z)M  for  all  u  G 

Validity  of  /  =  g  is  defined  accordingly,  hence,  l=/  =  giffl=/<g  and  \=  g  <  f.  As  consequence 
relation  on  formulas,  we  use  the  (global)  consequence  relation  that  we  define  as  follows  (similarly 
when  some  of  the  formulas  are  /,  =  gfi'. 

h  <  gi,  ■  ■  ■ ,  fn  <  gn  l=  /  <  g 
iff  N  A  <  91,  ■  ■  ■ ,  N  fn  <  gn  implies  N  f  <  g 

Also  f  \  <  gi, . . . ,  fn  <  N  /  <  5  holds  pathwise  if  it  holds  for  each  cu  G  fl 


6  Stochastic  Calculus 


In  this  section,  we  review  important  results  from  stochastic  calculus  [KS91,  0ksO7,  KP10]  that 
we  use  in  our  proof  calculus.  To  indicate  the  probability  law  of  process  X  starting  at  X0  =  x 
a.s.,  we  write  Px  instead  of  P.  By  Ex  we  denote  the  expectation  operator  for  probability  law 
Px.  That  is  Ex(f(Xt ))  :=  fn  f(Xt(uj))dPx(uj)  for  each  Borel-measurable  function  /  :  Rd  — >  M. 
A  very  important  concept  is  the  infinitesimal  generator  that  captures  the  average  rate  of  change  of 
a  process. 


Definition  4  (Infinitesimal  generator)  The  (infinitesimal)  generator  of  an  a.s.  right  continuous 
strong  Markov  process  ( e.g.,  solution  from  Def.  1 )  is  the  operator  A  that  maps  a  function  f  :  Rd  — *  M 
to  function  Af  :  Rd  — »  M  defined  as 


Af(x) 


|im  E*HXt)  -  f(x) 
t\ 0  t 


We  say  that  Af  is  defined  if  this  limit  exists  for  all  x  G  Wl.  The  generator  can  be  used  to  compute 
the  expected  value  of  a  function  when  following  the  process  until  a  Markov  time  without  solving 


the  SDE. 


Theorem  3  (Dynkin’s  formula  [0ksO7,  Theorem  7.4.1], [Dyn65,  p.  133])  Let  Xt  an  a.s.  right 
continuous  strong  Markov  process  (e.g.,  solution  from  Def.  1).  If  f  G  C'2(Wl,  M)  has  compact 
support  and  t  is  a  Markov  time  with  Ext  <  oo,  then 


Exf(XT )  =  f(x)  +  Ex  fT  Af(Xs)ds 

Jo 

Dynkin’s  formula  is  very  useful,  but  only  if  we  can  compute  the  generator  and  its  integral.  The 
generator  A  gives  a  stochastic  expression.  It  has  been  shown,  however,  that  it  is  equal  to  a  deter¬ 
ministic  expression  called  the  differential  generator  under  fairly  mild  assumptions: 
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Theorem  4  (Differential  generator  [0ksO7,  Theorem  7.3.3])  For  a  solution  Xt  from  Defi  1,  if 
f  £  C2(Md,  M)  is  compactly  supported,  then  Af  is  defined  and 

Af(x )  =  Lf(x )  :=  52bi(x)^(x)  + 


A  stochastic  process  y  that  is  adapted  to  a  filtration  {Pt)t> o  is  a  supermartingale  iff  E\Yt\  <  oo 
for  all  t  >  0  and 


E(Yt  |  J7,)  <  ys  for  all  t  >  s  >  0 


Proposition  1  (Doob’s  maximal  martingale  inequality  [KS91,  Theorem  1.3.8])  If  f(Xt)  is  a  cadlag 
supermartingale  with  respect  to  the  filtration  generated  by  (Xt)t>0  and  f  >  0  on  the  evolution  do¬ 
main  of  Xt,  then  for  all  A  >  0: 

P  sup  /( At)  >  A  I  < - 7 - 

\t>  0  /A 


7  Proof  Calculus 

Now  that  we  have  a  model,  logic,  and  semantics  for  stochastic  hybrid  systems,  we  investigate 
reasoning  principles  that  can  be  used  to  prove  logical  properties  of  stochastic  hybrid  systems. 
First  we  present  proof  rules  that  are  sound  pathwise,  i.e.,  satisfy  the  global  consequence  relation 
pathwise  for  each  oj  £  Q.  By  U  we  denote  the  binary  maximum  operator.  It  can  either  be  added 
into  the  language  or  approximated  conservatively  by  +  as  in  rule  (;).  Operator  U  coincides  with 
V  for  values  in  {0,1},  e.g.,  built  using  operators  A,  V,  -i,  (a)  from  characteristic  functions.  As  a 
supremum,  (a) B  only  takes  on  values  {0,1}  if  B  does. 

Theorem  5  (Pathwise  sound)  The  proof  rules  in  Fig.  3  are  globally  sound  pathwise. 

For  a  proof  see  Appendix B.l.  For  (;)',  (3  is  a.s.  continuous  at  0  if,  on  all  paths,  the  first  primitive 
operation  that  is  not  a  test  is  a  stochastic  differential  equation,  not  a  (random)  assignment.  Our 
rules  generalize  to  the  case  of  probabilistic  assumptions.  Note  that  formula  H  — >  f  <  A  in  mon'  is 
equivalent  to  Hf  <  H A  but  easier  to  read.  If  /  is  continuous,  rule  mon'  is  sound  when  replacing  the 
topological  closure  H  (which  is  computable  by  quantifier  elimination)  by  H,  because  the  inequality 
is  weak. 

Next  we  show  proof  rules  that  do  not  hold  pathwise,  but  still  in  distribution. 

Theorem  6  (Sound  in  distribution)  Rule  (©}  is  sound  in  distribution. 

P((Xa  ©  Vfi)f  eS)  =  A p({a)f  e  S)  +  uP((/3)f  e  S)  ((©)) 
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(x  :=0)f  =  fx  if  admissible  substitution  replacing  x  with  9  ((:=)) 

(7H)f  =  Hf  ((?)) 

(«;/?)/<  (a)  (/U  <J3)f)  (<(«)(/  +  </?>/)  if  0</)  ((;)) 

(a;  /3)f  <  (a)(/3)f  if  N  /  <  (/?)/  or  f3  continuous  at  0  a.s.  ((;)') 

(a)(Xf)  =  X(a)f  (()A) 

(a)(Xf +  ug)  <  X(a)f +  u(a)g  (()+) 

0  <  =  £>£>  <1  if  f?  boolean  from  characteristic  functions  (X) 

0  <  /  N  0  <  (a)/  (pos) 

f  <9  N  (off  <  (a) g  (mon) 

H  — >■  /  <  A  N  (dx  =  bdt  +  acibr  &  if)/  <  A  (A  e  Q)  (mon') 

{a)g  <g  1=  {a*)g  <  g  (ind) 


Figure  3:  Pathwise  proof  rules  for  ScLC 


For  a  proof  see  Appendix  B. 2.  How  to  prove  properties  about  random  assignment  x8  :=  *  depends 
on  the  distribution  for  the  random  assignment.  For  a  uniform  distribution  in  [0,1],  e.g.,  we  obtain 
the  following  proof  rule  that  is  sound  in  distribution: 

P({Xi:=*)f  eS)=  f  T{xi:^)fesdr  ((*)) 

Jo 

The  integrand  is  measurable  for  measurable  S  by  Corollary  1 .  The  rule  is  applicable  when  /  has 
been  simplified  enough  using  other  proof  rules  such  that  the  integral  can  be  computed  after  using 
(:=)  to  simplify  the  integrand. 

Theorem  7  (Soundness  for  stochastic  differential  equations)  If  function  f  e  C2(R(I  ■  M)  has  com¬ 
pact  support  on  H  (which  holds  for  all  f  G  M)  ifH  represents  a  bounded  set),  then  the  proof 

rule  (')  is  sound  for  X  >  0,p  >  0 

(ct)(H  — »  f)  <  Ap  H f>0  H^Lf<0 
(  ^  P((a)(dx  —  bdt  +  odW  &  H)(  >  A)  <  p 

Proof:  Since  f  has  compact  support  on  H,  it  has  a  C2(Mrf,  M)  modification  with  compact  support 
on  that  still  satisfies  the  premises  of  ('),  because  all  properties  of  f  in  the  premises  assume  H.  To 
simplify  notation,  we  write  f  (x)  for  [f]:t .  Let  X,  be  the  stochastic  process  [fix  =  bdt  +  odW  &  Hjz . 
Let  Xt  be  Xt  restricted  to  H,  i.e.,  the  stopped  process  Xt  :=  Xtnqdx=bdt+r7dW & z,  which  is 
stopped  at  a  Markov  time  by  Theorem  1.  The  stopped  process  Xt,  thus,  inherits  cadlag  and  strong 
Markov  properties  from  Xt;  see,  e.g.,  [Dyn65,  Theorem  10.2].  If  Af  is  defined  and  continuous 
and  bounded  on  H  [Dyn65,  Ch  11.3][Kus67,  Ch  1.3, 1.4],  then  the  infinitesimal  generator  of  Xt 
agrees  with  the  generator  of  Xt  on  II  (and  is  zero  otherwise).  This  is  the  case,  since  f  G  C2(WI ,  M) 
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has  compact  support  (thus  bounded  as  continuous),  because  Af  is  then  defined  and  Af  =  Lf  by 
Theorem  4,  hence,  L f  is  continuous,  because  b,  a  are  continuous  by  Def.  1. 

All  premises  of  rule  (')  still  hold  when  assuming  the  topological  closure  H  instead  of  H, 
because  the  functions  f  and  Lf  are  continuous  and  the  conditions  are  weak  inequalities,  thus, 
closed.  Consider  any  x  G  Rd  and  any  time  s  >  0.  The  deterministic  time  s  is  a  (very  simple) 
Markov  time  with  Exs  =  s  <  oo.  Since  f  is  compactly  supported,  Theorem  3  is  applicable  and 
implies  that 

Exf(Xs)  =  f(x)  +  Ex  [  Af(Xr)dr  (2) 

Jo 

Now  Lf  <  0  on  H  by  the  third  premise.  Hence,  Af  <  0  on  H,  because  Lf  =  Af  (on  H )  by  Theo¬ 
rem  4,  as  f  G  C2(Rd ,  R)  has  compact  support.  Because  X  and  X  have  a.s.  continuous  paths  and  are 
not  defined  on  the  event  {Z  H},  we  know  that  Xs  stays  in  the  closure  H  a.s.  Thus,  Af(Xs)  <  0 
a.s.,  hence,  fQs  Af(Xr)dr  <  0  a.s.,  thus,  Ex  f*  Af(Xr)dr  <  0.  Then  (2)  implies  Exf(Xs )  <  f(rc) 
for  all  x. 

Because  the  filtration  is  right-continuous  and  f  G  C  (Rd,  R)  is  compactly  supported  (hence 
bounded),  the  strong  Markov  property  [KS91,  Prop  2.6.7]  for  Xt  implies  for  all  t  >  s  >  0  that 
Px- a.s.:  Ex(f(Xt)\Es)  =  E^sf(Xt_s)  <  f(Xs).  The  inequality  holds,  since  Exf(Xs )  <  f(x)  for 
all  x,  s.  Thus,  f(Xt)  is  a  supermartingale  with  respect  to  Xt,  because  it  is  adapted  to  the  filtration 
of  Xt  (as  f  G  C'2(Rd,  M))  and  Ex\f(Xt)\  <  oo  for  all  t  since  f  G  C2(Rd,  R)  has  compact  support. 
Further,  f  (Xt)  inherits  continuity  from  Xt  (which  follows  from  Xt),  since  f  is  continuous. 

Thus,  by  the  second  premise,  Proposition  1  is  applicable.  Consider  any  initial  state  Y  :=  [a]f 
for  X.  Thus,  P  (supt>0  f(Xt)  >  X  \  Eq)  <  Ef<£  ]  by  Proposition  1  (filtration  at  X0  is  E0).  On  event 
{Y  y=-  II},  X  is  not  defined  and  nothing  to  show.  On  {Y  \=  II},  f(  Y)  <  X p  is  valid  where  relevant 
by  the  first  premise.  This  implies  the  conclusion,  as  \{dx  =  bdt  +  adW  &  H) ff  =  supi>0  ((Xj). 

□ 

The  implications  in  the  premises  can  be  understood  like  that  in  mon' .  Let  H  be  given  by  first-order 
real  arithmetic  formulas.  If  f  is  polynomial  and,  thus,  f  G  C2(Rd,  R),  then  the  second  and  third 
premise  of  (')  are  in  first-order  real  arithmetic,  hence  decidable.  Note  that  our  proof  rules  can  be 
generalized  to  probabilistic  assumptions  by  the  rule  of  partition  and  then  combined. 

The  proof  shows  that  it  is  enough  to  assume  the  first  premise  holds  only  a.s.  From  the  proof 
we  see  that  it  would  be  sufficient  to  replace  the  third  premise  of  (')  with  J}}  Lf(Xr)dr  <  0.  This 
is  a  weaker  condition,  because  it  does  not  require  Lf  <  0  always,  but  only  “on  average”.  But  this 
condition  is  computationally  more  involved,  because  the  integral  needs  to  be  computed  first.  For 
polynomial  expressions,  this  is  not  too  difficult,  but  still  increases  the  polynomial  degree. 

A  simple  two-dimensional  example  is  the  following  for  H  =  x2  +  y2  <  10: 

P{{7x2+y2<^x:=^]dx  =  ^dt-ydW,dy  =  ^-dt  +  xdW  &  H)x2+y2>l)  <  ^ 
which  can  be  proven  easily  using  (;)',  (;)  (?),  (:=),  ('),  since  f  =  x2  +  y2  >  0  and 


Lf 


1  f  Of  8f  2d2f  n  d2f  o  d2f  \ 

2  V  Xdx  Vdy+y  dE~  XVdJJdy  +X  fry2) 


<  0 
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This  implies  the  second  and  third  premise  of  (').  In  order  to  see  why  the  first  premise  holds  and 
how  the  property  can  be  concluded,  we  first  look  at  a  simpler  example. 


P((7x2  +  y2  <  dx  =  —  ^dt  —  ydW ,  dy  =  —  '^dt  +  xdW  k,H)x2  +  y2  >  1)  <  ^ 

O  Zj  Zj  o 

The  second  and  third  premise  of  (')  continue  to  hold  for  this  simpler  example.  We  conclude  the 
first  premise  of  (')  using  (?) 

(7x2  +  y2  <  \){H  ->•  f)  =  (h  ->•  x2  +  y2  <  ^  (x2  +  y2)  <  1  *  \ 


Hence,  (')  is  applicable  implying  the  conclusion 

P((7x2  +  y2  <  dx  =  —  —dt  —  ydW. ,  dy  =  —  ^ dt  +  xdW  &  H)x2  +  y2  >  1) 

C)  Zj  z 

Using  (;)'  inside  the  probability,  this  expression  is  <  the  following 

P((lx2  +  y2  <  -){dx  =  —  ~dt  —  ycil-U,  dy  =  —  ^-dt  +  a;dIU  k,Pd)x2  +  y2  >  1)  <  - 
o  z  z  o 


In  the  same  way,  we  can  prove  the  original  property: 

1  1 

P({lx2  +  y2  <  x  :=  dx  =  ~°^dt  —  ycilU,  dy  =  —  | dt  +  a;dIU  H)x2  +  y2  >  1)  <  - 

The  only  change  is  as  follows.  By  (;)  we  conclude 


(?a;2  +  y2  <  a;  :=  f)  <  (?x2  +  y2  <  \)({H  ->■  f)  U  (x  :=  |)(Tf  f)) 


which,  by  (:=),  is  <  the  following,  because  x |  makes  the  f-value  drop  (and  ?x2  +  y2  <  | 
implies  H  even  after  x  :=  |): 

(?a:2  +  y2  <  ^){H  f)  =  -)•  a:2  +  y2  <  ^  (: x 2  +  y2)  <  1  *  ^ 


The  arithmetic  is  easily  decidable  by  quantifier-elimination  in  real-closed  fields. 


8  Related  Work 

Our  approach  is  partially  inspired  by  the  work  of  Kozen,  who  studied  3  semantics  of  programs 
with  random  number  generators  [Koz81]  and  probabilistic  PDL  [Koz85].  We  generalize  from 
discrete  systems  to  stochastic  hybrid  systems.  To  reflect  the  new  challenges,  we  have  departed 
from  probabilistic  PDL.  Kozen  uses  a  measure  semantics.  We  choose  a  semantics  that  is  based  on 
stochastic  processes,  because  the  temporal  behavior  of  SHPs  is  more  crucial  than  that  of  abstract 
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discrete  programs.  Sd£  further  uses  a  supremal  semantics  that  is  more  interesting  for  stochastic 
worst-case  verification  than  the  integral  semantics  assumed  in  [Koz85]. 

The  comparison  to  a  first-order  dynamic  logic  for  deterministic  programs  with  random  num¬ 
ber  generators  [FH84]  is  similar.  They  axiomatize  relative  to  first-order  analysis  with  arithmetic, 
enriched  with  frequencies  and  random  number  generators.  They  do  not  show  how  this  logic  could 
be  handled  (incompletely). 

Our  approach  for  stochastic  differential  equations  is  inspired  by  barrier  certificates  [PJP07].  We 
extend  this  work  by  identifying  the  assumptions  that  are  required  for  soundness  of  using  Dynkin- 
type  arguments  for  stochastic  differential  equations.  They  propose  to  use  global  generators  for 
switching  diffusion  processes  (which  cannot  reset  variables).  We  use  logic  and  compositional 
proofs  for  SHPs. 

Probabilities  and  logic  have  also  been  used  in  AI,  e.g.,  [RD06].  Markov  logic  networks  are 
a  combination  of  Markov  networks  and  first-order  logic  and  resembles  logic  programming  with 
weights  for  probabilities.  They  are  restricted  to  finite  domains,  which  is  not  the  case  in  stochastic 
hybrid  systems. 

Model  checking  has  been  used  for  discrete  probabilistic  systems  like  finite  Markov  chains,  e.g., 
[YKNP06],  and  probabilistic  timed  automata  [KNSW07].  Assume-guarantee  model  checking  is  a 
challenge  for  discrete  probabilistic  automata,  with  recent  successes  for  finite  automata  assumptions 
[KNPQ10].  We  use  a  compositional  proof  approach  based  on  logic  and  consider  stochastic  hybrid 
systems. 

Statistical  model  checking  has  been  suggested  for  validating  stochastic  hybrid  systems  [MS06] 
and  later  refined  for  discrete-time  hybrid  systems  with  a  probabilistic  simulation  function  [ZPC10] 
based  on  corresponding  discrete  probabilistic  techniques  [YKNP06].  They  did  not  show  mea¬ 
surability  and  do  not  support  stochastic  differential  equations  [ZPC10].  Validation  by  simulation 
is  generally  unsound,  but  the  probability  of  giving  a  wrong  answer  can  sometimes  be  bounded 
[YKNP06,  ZPC10]. 

Franzle  et  al.  [FTE10]  show  first  pieces  for  continuous-time  bounded  model  checking  of  prob¬ 
abilistic  hybrid  automata  (no  stochastic  differential  equations). 

Bujorianu  and  Lygeros  [BL06]  show  strong  Markov  and  cadlag  properties  for  a  class  of  systems 
known  as  General  Stochastic  Hybrid  Systems.  They  also  study  an  interesting  concatenation  opera¬ 
tor.  For  an  overview  of  model  checking  techniques  for  various  classes  of  stochastic  hybrid  systems, 
we  refer  to  [CL06].  Most  verification  techniques  for  stochastic  hybrid  systems  use  discretizations, 
approximations,  or  assume  discrete  time,  bounded  horizon  [KR08,  CL06,  APLS08,  HLSOO].  We 
consider  the  continuous-time  behavior  and  develop  compositional  logic  and  theorem  proving. 


9  Conclusions 

We  introduce  the  first  verification  logic  for  stochastic  hybrid  systems  along  with  a  compositional 
model  of  stochastic  hybrid  programs.  We  prove  theoretical  properties  that  are  important  for  well- 
definedness  and  measurability  and  we  develop  a  compositional  proof  calculus.  Our  logic  makes  the 
complexity  of  stochastic  hybrid  systems  accessible  in  logic  with  simple  syntactic  proof  principles. 
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Our  results  indicate  that  Sd£  is  a  promising  starting  point  for  the  study  of  logic  for  stochastic 

hybrid  systems.  Extensions  include  nondeterminism. 
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A  Proofs  for  Semantics 


In  the  appendices,  we  provide  proofs  for  the  results  in  this  paper.  In  this  appendix,  we  provide 
proofs  for  the  semantics  and  its  well-definedness. 

A.l  Proof  of  Adaptive  Cadlag  Process  with  Markov  Times 

Prooffof  Theorem  1):  We  prove  cadlag,  adaptedness,  and  Markov  time  properties  simultaneously 
by  induction  on  the  structure  of  a.  These  parts  partially  depend  on  each  other,  so  we  prove  them 
together  not  separately.  To  simplify  notation,  we  shift  time  so  that  processes  start  at  time  0. 

1-3.  Deterministic  times  (jay  :=  9\)Z  =  (\xt  :=  *\)Z  =  (\?H  |)z  =  0  are  trivial  Markov  times.  Fur¬ 
thermore,  the  process  \xi  :=  9]Z  is  adapted  to  the  filtration  generated  by  Z.  Process  [[?  II  || z 
is  also  adapted  if  it  is  defined  (otherwise  there  is  nothing  to  show).  Similarly,  \xi  :=  *}Z  is 
adapted  to  the  filtration  generated  by  Z  and  the  u.i.i.d.  random  variable  ( [[a;,.  :=  *] z)j  =  Ui. 
Process  [?TF|Z  is  cadlag  (even  constant)  if  it  is  defined,  otherwise  there  is  no  continuity 
question  (can  be  considered  stuck  at  absorbing  state).  Processes  {xi  :=  9]z  and  \xi  :=  *]Z 
are  trivially  cadlag  (even  continuous)  as  the  time  domain  is  {0}. 

4.  (| dx  =  bdt  +  crdW  &  H\)Z  =  inf (t  >  0  :  Xt  qL  H}  is  a  Markov  time  when  H  is  any  Borel 
set  [0ksO7,  Ex  7.2.2][Dyn65,  Vol.  II,  4.5.C.e],  since  we  complete  the  filtration  to  include 
all  null  sets.  Here  Xt  is  the  process  {dx  =  bdt  +  adW  &  II\\Z ■  More  generally,  for  pro¬ 
gressively  measurable  processes  like  right-continuous  adapted  processes,  the  hitting  time 
of  a  measurable  set  is  a  Markov  time  by  the  (deep)  debut  theorem  [0ksO7].  Solutions  of 
stochastic  differential  equations  are  adapted  to  the  filtration  generated  by  ( Ws)s<t  and  Z 
[0ksO7,  Th  5.2.1][KP10,  Ch  4.5]  and  have  almost  surely  continuous  paths  by  a  consequence 
of  Kolmogorov’s  continuity  theorem  [0ksO7,  Th  2.2.3]. 

5.  By  induction  hypothesis,  (\a\)Z  is  a  Markov  time,  hence  Tu< \(\a\)z  is  a  Markov  time,  since 
the  filtration  includes  U  and  the  indicator  function  only  takes  on  values  0  (where  0  is  a  stop- 

z  z 

ping  time)  or  1  (where  l(|a|)  is  a  Markov  time).  Similarly  lu>\(\/3\)  is  a  Markov  time.  As 
the  sum  of  two  Markov  times,  (Aa  ©  uj3 \)z  is  a  Markov  time  [KS91,  Lem  1.2.9].  Because 
cadlag  functions  form  an  algebra  (Skorokhod  space),  the  linear  combination  [A a:  ©  v/3}z  is 
cadlag  by  induction  hypothesis  for  every  outcome  of  U.  This  linear  combination  is  adapted, 
because,  by  induction  hypothesis,  the  parts  are  adapted  and  the  choice  U  generates  the  filtra¬ 
tion. 

6.  By  induction  hypothesis,  {a}2  is  adapted  to  and  (] a  |) z  a  Markov  time  for  the  filtration  {T't  )t>0 

generated  by  Z  and  the  constituent  Brownian  motion  and  uniform  processes  during  a.  Es¬ 
pecially,  is  a  random  variable.  By  induction  hypothesis,  ^“l)Z  j  is,  thus, 

adapted  to  and  a  Markov  time  for  the  filtration  generated  by 

and  the  constituent  Brownian  motion  and  uniform  processes  during  f3.  With  a  time  shift 
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by  — (M)z,  ^ z  is  then  adapted  to  the  filtration  Especially, 

(Tt)t>o  already  includes  (Jy)t>o  and  the  time-shifted  (F”  ^t^)tx\n\)z ■  Note  that  random 
variable  [a]f[)Z  does  not  contribute  to  this  filtration,  because  it  is  already  J^z-measurable 
by  induction  hypothesis.  Consequently,  [a;  /3]z  is  adapted  to  (Ft)t> o,  because  both  of  its 

cases,  [ a]z  and  ,  are  adapted  and  the  condition  which  case  applies  is  an  event  of  a 

Markov  time.  Similarly,  fla;;  f3\)Z  =  (\a\)Z  +  (|/3[)^“!>z  is  a  sum  of  two  Markov  times  and, 
thus,  a  Markov  time  [KS91,  Lem  1.2.9]. 

By  induction  hypothesis,  [a;/3]z  is  cadlag  on  [0,  (|a|)z)  and  on  ((|q;[)z,  oo),  because  the 
constituent  fragments  are.  At  (\a\)z ,  process  [a;  f3}z  is  cadlag,  by  construction  (it  is  defined 
in  terms  of  [3  on  the  left-closed  interval  [(H)  ,oo),  hence  cadlag  even  if  there  is  a  jump 
before  /3  starts). 

7.  Because  (|an|)z  are  increasing,  (| a*\)Z  =  lim)WOO  (\o:n\)Z  =  supn>1  (\an\)Z ,  which  is  a  Markov 
time  [KS91,  Lem  1.2.11],  since,  by  induction  hypothesis,  the  (|a"|)z  are  Markov  times.  Pro¬ 
cess  {a*}Z  is  adapted,  because  for  each  t,  the  constituent  process  \oin\Z  is  adapted  on  each 
event  {(|an|)z  >  t}  by  induction  hypothesis.  Note  that  \a*\z  is  not  defined  if  this  never 
happens,  i.e.,  on  the  event  {in  (\an\)z  >  t}.  Since  the  value  \a*\z  is  defined  on  an  n  that 
satisfies  the  open  event  {(\oin\)Z  >  t},  the  process  is  cadlag  as  long  as  it  is  defined. 

□ 


A.2  Proof  of  Measurability 

Proof(of  Theorem  2):  We  need  to  show  that  \f\z  is  measurable  as  a  function  of  uj  e  Q.  Wc 
prove  this  by  induction  on  the  structure  of  /. 

1 .  \F\Z  =  F\Z)  is  a  random  variable,  because  Z  is  measurable  and  Fe  is  Borel(!)-measurable. 
Thus,  the  composition  FC(Z)  is  measurable  (the  rr- algebras  in  the  composition  are  compati¬ 
ble). 

2.  [A {  Fvg\Z  =  A[/]z  +  ulg]z  is  a  linear  combination,  hence,  measurable  by  induction 
hypothesis,  because  measurable  functions  form  an  algebra. 

3.  =  [B\z  *  \f\z  is  a  product,  hence,  measurable  by  induction  hypothesis,  because 
measurable  functions  form  an  algebra. 

4.  [(a)/]z  =  sup{[/]Wt  :  0  <  t  <  (|a|)z}  is  measurable  for  the  following  reason.  By  The¬ 
orem  1,  \a\z  is  measurable  (adapted).  By  induction  hypothesis,  [/J^4  is  measurable  for 
each  t.  We  need  to  show  that  the  supremum  is  still  measurable.  Unfortunately,  suprema  of 
measurable  functions  over  uncountable  sets  are  generally  not  measurable.  Yet,  the  (point- 
wise)  supremum  of  a  countable  sequence  of  measurable  functions  is  measurable  [Wal95, 
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§9.9].  Consider  a  rational  mesh  n  :=  {U,  t2,  ■  ■  ■ ,  tn}  C  Q  with  times  0  <  U  <  ■  ■  ■  <  tn. 
By  induction  hypothesis,  J/]^*  is  measurable  for  each  t  E  it.  Hence,  the  (finite)  count¬ 
able  supremum  sup{[/]^‘  :  t  E  ir,  t  <  (|  or  D  ^ }  is  measurable  (as  a  pointwise  function  of 
lu  E  H).  Unlike  the  set  of  infinite  sequences  in  Q,  the  set  of  finite  sequences  in  Q  is  count¬ 
able.  Thus,  the  countable  supremum  sup{[/]^*  :  t  <  (| af  ,t  E  tt  for  a  rational  mesh  n} 
is  measurable,  because  the  set  of  rational  meshes  is  countable.  In  general,  however,  this  lat¬ 
ter  supremum  does  not  coincide  with  the  supremum  defining  [[(a)/]Z.  But  since  \af  is  also 
cadlag  a.s.  by  Theorem  1,  they  do  coincide  (each  path  is  a.s.  right-continuous).  Note  that 
either  left  or  right  continuity  would  be  sufficient  to  ensure  that  there  is  a  convergent  sequence 
of  rational  meshes  whose  values  converge  to  the  value  at  each  real  point  in  the  interval.  Note, 
however,  that  this  only  gives  us  information  about  the  supremum  on  0  <  t  <  (|a  [)Z  for  a  right 
continuous  process,  because  (|o;|)  could  be  irrational  and  no  convergent  sequence  of  rational 
points  ti  >  (| a  |) z  from  the  right  is  in  the  interval.  But,  when  taking  the  (binary)  pointwise 

supremum  of  [/]^(|a|)Z  and  the  above  supremum,  we  obtain  the  desired  equality. 


□ 


B  Soundness  Proofs 

In  this  appendix,  we  provide  proofs  for  the  soundness  theorems. 


B.l  Proof  of  Pathwise  Global  Soundness 


Proof(of  Theorem  5):  We  prove  that  the  rules  are  globally  sound  pathwise  (which  coincides  with 
locally  sound  if  they  have  no  assumptions)  by  showing  that  they  hold  for  any  M,d- valued  random 
variable  Z  pathwise,  i.e.,  on  every  path  for  every  uj  E  Cl. 

(:=)  Soundness  of  rule  (:=)  is  similar  to  classical  dynamic  logic  [Pra76].  That  is,  [(a:  :=  9)f\z  = 

Uy][.t.=6>]°  _  deterministically  (for  all  u>  E  U).  Note  that  the  supremum  disappears, 

because  of  (|x  :=  9 \)z  =  0. 

(?)  {HfY  =  {H\Z  *  Iff  is  equal  to 


[<?tf)/f 


sup{[/j™  :  0  <  t  <  (|? H\f } 


Iff  on  event  {Z  |=  H} 
0  on  event  {Z  H } 


because  (]  ?  // 1) Z  =  0  (on  all  events)  and  our  convention  evaluates  all  function  terms  /  to  0  in 
undefined  states  (on  the  event  that  1H  fails  by  {Z  ^  II}). 


(;) 


I(o;/J)/f  =  sup{  [/]>“•«?  :  0  <  r  <  fla;  pf  =  Also  [<«)(/  U  (/?)/)f 

SUP{[/  L-1  (/?)/!  :  0  <  t  <  (|a|)Z}.  The  latter  equals  sup  {[/]^*  U  sup{[/]^s  :  0  < 
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S  <  d/3|)Wt  }  ;  o  <  t  <  d « I) ^ } .  With  these  expansions,  [(a;/3)/]z  <  |(a)(/U  (/3}f)Jz 

holds  as  follows.  For  each  path,  the  values  of  |[/]^Q’^r  on  the  event  {r  >  (|a|)z}  are  included 
in  the  nested  supremum  for  {{a)(f  U  (/3)f)j  by  choosing  t  ■=  (|cr D  ,s:=r-  (]a|)z  '.  The 
values  of  [/]  on  the  event  {r  <  (|  a  [) Z  }  are  included  in  the  nested  supremum  by  choos¬ 
ing  t  r  and  the  left  side  of  the  maximum  [/]  LI  ...  in  the  expression.  Note  that  the  two 
sides  are  generally  not  equal,  because  a  has  to  run  to  completion  before  ft  starts  in  («;£>/. 
but  a  can  stop  early  in  (a)  (/  U  (/3)f)  and  [3  can  then  start  already. 

If,  in  addition,  NO  <  /,  then  N  0  <  {{3)f  by  pos.  Hence,  mon  implies  by  the  semantics  of  U 

that  N  (a;P)f  <  (a)(f  U  (/3)f)  <  (a)(f  +  (/?)/). 

(;)'  If  N  /  <  (/?)/,  then  (;)'  follows  from  (;)  directly.  If,  instead,  [/3]p  is  continuous  at  0  a.s., 
then  the  proof  for  (;)  does  not  need  / U.  It  can  use  t  :=  r,  s  :=  0  on  the  event  {r  <  (|ct|)  }, 
because  the  process  for  f3  a.s.  will  not  change  the  value  of  /  at  time  0  (a.s.  continuity).  The 
proof  of  (;)  for  event  (r  >  (jG:|)Z}  does  not  use  / U  and  carries  over  to  (;)'  directly. 

()A  i(a)(\f)Jz  =  sup{[A/]Wt  :  0  <  t  <  (\a\)z}  =  sup{A[/]Wt  :  0  <  t  <  {\a\)z}  = 
A[(«)/f. 

()+  1(a) (Xf  +  vg)]Z  =  sup{[A/  +  :  0  <  t  <  (|a|)z}.  This  is  equal  to  sup{A[/]^*  + 

vlgfaj*  :  0  <  t  <  (| cr D ^ }  <  \\(a)f\z  +  u\(a)g\z .  The  two  sides  are  not  equal  if  the 
suprema  [(a)/]z  and  l(o)g\/  are  at  different  times. 


IB  is  a  Boolean  combination  of  characteristic  functions  of  measurable  sets.  Characteristic 
functions  only  take  on  the  values  0  or  1,  for  which  X  holds.  Boolean  combinations  preserve 
this  property. 

pos  Rule  pos  is  derivable  from  mon  and  ()A.  By  mon,  0  <  /  N  (a)0  <  ( a)f .  By  mon,  (a)  0  =  (a)(0  *  0)  =  0(a) 

mon  LetN  /  <  g,  i.e.,  If]'1  <  fry) '  for  all  V.  Hence,  by  Theorem  1 ,  for  random  variable  V  ■■=  H  f, 
we  get  [/]  <  \g |  .  Since  t  is  arbitrary,  this  implies 


[(«)/r  =  sup{[/lN‘  :  0  <  t  <  (| a[)z}  <  sup{[c/]M*  :  0  <  t  <  (| a\)z}  =  [(a 


Hi 


Hence,  [(a)/]z  <  [(a)(y]z,  which  implies  mon  since  Z  was  arbitrary. 

ind  Assume  N  (a)g  <  g,  which  implies  N  (a;  a)g  <  { a)(g  U  (a)g)  =  (a)g  <  g  by  (;).  By  in¬ 
duction,  N  (an)g  <  g.  Since  tz£  N  was  arbitrary,  we  get  N  ( a*)g  <  g. 


??  Assume  N  0  <  /  and  N  0  <  /  +  ( a)g  <  g.  First  note  that  N  0  <  /  +  ( a)g  <  g  directly  im¬ 
plies  NO  <  g,  which  implies  0  <  (a)g  by  pos,  which  implies  N  /  <  g  using  N  0  <  /  +  (a)g  <  g. 
Therefore,  mon  implies  N  ( a*)f  <  ( a*)g .  Now,  N  0  <  /  and  N  0  <  /  +  (a)g  <  g  together 
imply  N  (a)g  <  g.  Hence,  ind  implies  N  (a*)g  <  g.  Together  with  N  ( a*)f  <  ( a*)g ,  this 
implies  N(cC)/<yy. 
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mon'  Assume  N  H  — y  f  <  A.  Let  Xt  be  the  stochastic  process  \dx  =  bdt  +  adW  &  HJZ .  Let  Xt 
be  Xt  restricted  to  H,  i.e.,  Xt  :=  Xtnndx=bdt+crdW  hH*z,  which  is  stopped  at  a  Markov  time  by 
Theorem  1.  Because  X  (and,  thus,  X)  have  a.s.  continuous  paths  and  are  not  defined  on  the 
event  { Z  //},  we  know  that  Xs  stays  in  the  closure  H  a.s.  Thus,  Xt[t]  \=  H  a.s.  for  all  t. 
Hence,  by  assumption  [/J <  A  for  all  t.  Then  [(dx  =  bdt  +  adW  &  H)fjz  <  A  =  [XJZ . 

□ 

B.2  Proof  of  Soundness  in  Distribution 

Prooffof Theorem 6):  [(Aa  0  v(3)ff  =  sup {[ffu<^+Xu>^t  :  0  <t<  (|Aa  ©  v/3\)z}, 
with  (jAa  ©  af3\)Z  =  Tu<\(\oi\)Z  +lu>\(\/3\)z .  This  expression  splits  into  two  disjoint  events,  one 
with  {U  <  A}  and  one  with  {U  >  A}.  Thus,  by  additivity  for  disjoint  events: 

P([(Aa  ©  u/3)f\z  e  S) 

=  P(U  <  A,  sup{[/]W"  :  0  <  t  <  (I a\)z}  e  S ) 

+  P(U  >  A,  sup{[/]^4  :  0  <  t  <  (|/3[)Z}  G  S )  cr-additive 

=  P(U  <  A,  {(a) ff  G  S)  +  P(U  >  A,  mff  G  S) 

=  P(U  <  A )P({(a)ff  G  S)  +  P(U  >  A )P(\((d)ff  G  S)  independent 

=  A P(l(a)ff  ES)  +  uP(mff  G  S)  A  +  fi  =  1 

□ 
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